User interface interaction using touch input force

ABSTRACT

Providing a user interface interaction is disclosed. A first set of one or more indicators associated with a first location on a touch input surface and a first force intensity of a touch input provided on the touch input surface is received. A user interface object associated with the first location is determined. A second set of one or more indicators associated with a second location on the touch input surface and a second force intensity of the touch input that has been in continuous contact with the touch input surface at least since the touch input was provided at the first location is received. The user interface interaction based at least in part on the user interface object, the first set of indicators, and the second set of indicators is provided.

CROSS REFERENCE TO OTHER APPLICATIONS

This application claims priority to U.S. Provisional Patent ApplicationNo. 61/561,660 entitled TOUCH SCREEN SYSTEM UTILIZING ADDITIONAL AXISINFORMATION FOR SELECTED APPLICATIONS filed Nov. 18, 2011 which isincorporated herein by reference for all purposes.

This application claims priority to U.S. Provisional Patent ApplicationNo. 61/561,697 entitled TOUCH SCREEN SYSTEM UTILIZING ADDITIONAL AXISINFORMATION filed Nov. 18, 2011 which is incorporated herein byreference for all purposes.

This application claims priority to U.S. Provisional Patent ApplicationNo. 61/673,102 entitled UTILIZING TOUCH PRESSURE INFORMATION INGRAPHICAL USER INTERFACES filed Jul. 18, 2012 which is incorporatedherein by reference for all purposes.

BACKGROUND OF THE INVENTION

Various technologies have been used to detect a touch input on a displayarea. The most popular technologies today include capacitive andresistive touch detection technology. Using resistive touch technology,often a glass panel is coated with multiple conductive layers thatregister touches when physical pressure is applied to the layers toforce the layers to make physical contact. Using capacitive touchtechnology, often a glass panel is coated with material that can hold anelectrical charge sensitive to a human finger. By detecting the changein the electrical charge due to a touch, a touch location can bedetected. However, with resistive and capacitive touch detectiontechnologies, the glass screen is required to be coated with a materialthat reduces the clarity of the glass screen. Additionally, because theentire glass screen is required to be coated with a material,manufacturing and component costs can become prohibitively expensive aslarger screens are desired.

Another type of touch detection technology includes surface acousticwave technology. One example includes the Elo Touch Systems AcousticPulse Recognition, commonly called APR, manufactured by Elo TouchSystems of 301 Constitution Drive, Menlo Park, Calif. 94025. The APRsystem includes transducers attached to the edges of a touchscreen glassthat pick up the sound emitted on the glass due to a touch. However, thesurface glass may pick up other external sounds and vibrations thatreduce the accuracy and effectiveness of the APR system to efficientlydetect a touch input. Another example includes the Surface AcousticWave-based technology, commonly called SAW, such as the Elo IntelliTouchPlus™ of Elo Touch Systems. The SAW technology sends ultrasonic waves ina guided pattern using reflectors on the touch screen to detect a touch.However, sending the ultrasonic waves in the guided pattern increasescosts and may be difficult to achieve. Detecting additional types ofinputs, such as multi-touch inputs, may not be possible or may bedifficult using SAW or APR technology.

Additionally, current touch detection technology cannot reliably,accurately, and efficiently detect pressure or force of a touch input.Although prior attempts have been made to detect pressure of touch inputby measuring the relative size of a touch input (e.g., as a fingerpresses harder on a screen, area of the finger contacting the screenproportionally increases), these attempts produce unreliable resultswhen a hard stylus or different sized fingers are used. Therefore thereexists a need for a better way to detect an input on a surface. Onceforce or pressure of a touch input can be reliably detected, userinterface interaction utilizing force or pressure may be provided.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments of the invention are disclosed in the followingdetailed description and the accompanying drawings.

FIG. 1 is a block diagram illustrating an embodiment of a system fordetecting a surface disturbance.

FIG. 2 is a block diagram illustrating an embodiment of a system fordetecting a touch input.

FIG. 3 is a flow chart illustrating an embodiment of a process forcalibrating and validating touch detection.

FIG. 4 is a flow chart illustrating an embodiment of a process fordetecting a user touch input.

FIG. 5 is a flow chart illustrating an embodiment of a process fordetermining a location associated with a disturbance on a surface.

FIG. 6 is a flow chart illustrating an embodiment of a process fordetermining a location associated with a disturbance.

FIG. 7 is a flowchart illustrating an embodiment of a process ofdetermining a force associated with a user input.

FIG. 8 is a flowchart illustrating an embodiment of a process fordetermining an entry of a data structure used to determine a forceintensity identifier.

FIG. 9 includes graphs illustrating examples of a relationship between anormalized amplitude value of a measured disturbance and an appliedforce.

FIG. 10 is a flowchart illustrating an embodiment of a process forproviding a combined force.

FIG. 11 is a flowchart illustrating an embodiment of a process forproviding a user interface interaction.

FIG. 12 is a diagram showing an example user interface interaction usingforce information to drag and drop an item into a file system folder.

FIG. 13 is a diagram showing an example user interface interaction usingforce information to provide a context menu.

FIG. 14 and FIG. 15 are diagrams showing examples of user interfaceinteractions using force information to navigate a menu.

FIG. 16 is a diagram showing an example user interface interaction usingforce information to interact with a virtual keyboard.

FIG. 17 and FIG. 18 are diagrams showing example user interfaceinteractions using force information to zoom and select user interfaceobjects.

FIG. 19 is a graph illustrating an example of a relationship betweendetected touch input force and direction of change in audio volume.

FIG. 20 is a diagram showing an example user interface interaction usingforce information to interact with a slider bar.

DETAILED DESCRIPTION

The invention can be implemented in numerous ways, including as aprocess; an apparatus; a system; a composition of matter; a computerprogram product embodied on a computer readable storage medium; and/or aprocessor, such as a processor configured to execute instructions storedon and/or provided by a memory coupled to the processor. In thisspecification, these implementations, or any other form that theinvention may take, may be referred to as techniques. In general, theorder of the steps of disclosed processes may be altered within thescope of the invention. Unless stated otherwise, a component such as aprocessor or a memory described as being configured to perform a taskmay be implemented as a general component that is temporarily configuredto perform the task at a given time or a specific component that ismanufactured to perform the task. As used herein, the term ‘processor’refers to one or more devices, circuits, and/or processing coresconfigured to process data, such as computer program instructions.

A detailed description of one or more embodiments of the invention isprovided below along with accompanying figures that illustrate theprinciples of the invention. The invention is described in connectionwith such embodiments, but the invention is not limited to anyembodiment. The scope of the invention is limited only by the claims andthe invention encompasses numerous alternatives, modifications andequivalents. Numerous specific details are set forth in the followingdescription in order to provide a thorough understanding of theinvention. These details are provided for the purpose of example and theinvention may be practiced according to the claims without some or allof these specific details. For the purpose of clarity, technicalmaterial that is known in the technical fields related to the inventionhas not been described in detail so that the invention is notunnecessarily obscured.

Detecting a force of a touch input is disclosed. In some embodiments, anacoustic transducer transmits an acoustic wave through a medium of atouch input surface. The acoustic wave may be scattered by the touchinput producing a scattered acoustic wave. An acoustic detector thatdetects the scattered acoustic wave and the acoustic detector outputs asignal indicating variation of the acoustic wave that is indicative ofan amount of force associated with the touch input. In some embodiments,the force of a touch input is associated with the amount of deflectionor movement of a touch surface medium caused by a touch input. Forexample, as a finger or stylus touches and pushes a touch input surfaceharder, the amount of force detected gets functionally larger as well.The pressure of a touch input is the force of touch input per unit areaof the touch input. For example, the total force of a touch inputdivided by the area of contact of the touch input equals the pressure ofthe touch input. Although force of a touch input is utilized in thespecification, pressure of a touch input may be used as well. In somecases, when a user pushes harder on a surface such as a touch screendisplay with a fingertip, the pressure of the touch input may staysubstantially constant because the size of the fingertip in contact withthe surface becomes larger due to the softness of the fingertip. Inorder to detect that the user is pushing harder on the surface, thetotal force of the touch input may be used instead of the pressure ofthe touch input. In some embodiments, a force of a touch input is usedto provide user interface interaction.

In some embodiments, a user touch input on the glass surface of adisplay screen is detected. In some embodiments, a signal such as anacoustic or ultrasonic signal is propagated freely through a propagatingmedium with a surface using a transmitter coupled to the medium. Whenthe surface is touched, the propagated signal is disturbed (e.g., thetouch causes an interference with the propagated signal). In someembodiments, the disturbed signal is received at a sensor coupled to thepropagating medium. By processing the received signal and comparing itagainst an expected signal without the disturbance, a location on thesurface associated with the touch input is at least in part determined.For example, the disturbed signal is received at a plurality of sensorsand a relative time difference between when the disturbed signal wasreceived at different sensors is used to determine the location on thesurface. In various embodiments, the touch includes a physical contactto a surface using a human finger, pen, pointer, stylus, and/or anyother body parts or objects that can be used to contact or disturb thesurface. In some embodiments, the touch includes an input gesture and/ora multi-touch input.

In some embodiments, the disturbed signal is used to determine one ormore of the following associated with a touch input: a gesture, acoordinate position, a time, a time frame, a direction, a velocity, aforce magnitude, a proximity magnitude, a pressure, a size, and othermeasurable or derived parameters. In some embodiments, by detectingdisturbances of a freely propagated signal, touch input detectiontechnology can be applied to larger surface regions with less or noadditional cost due to a larger surface region as compared to certainprevious touch detection technologies. Additionally, the opticaltransparency of a touch screen may not have to be affected as comparedto resistive and capacitive touch technologies. Merely by way ofexample, the touch detection described herein can be applied to avariety of objects such as a kiosk, an ATM, a computing device, anentertainment device, a digital signage apparatus, a cell phone, atablet computer, a point of sale terminal, a food and restaurantapparatus, a gaming device, a casino game and application, a piece offurniture, a vehicle, an industrial application, a financialapplication, a medical device, an appliance, and any other objects ordevices having surfaces.

FIG. 1 is a block diagram illustrating an embodiment of a system fordetecting a surface disturbance. In some embodiments, the system shownin FIG. 1 is included in a kiosk, an ATM, a computing device, anentertainment device, a digital signage apparatus, a cell phone, atablet computer, a point of sale terminal, a food and restaurantapparatus, a gaming device, a casino game and application, a piece offurniture, a vehicle, an industrial application, a financialapplication, a medical device, an appliance, and any other objects ordevices having surfaces. Propagating signal medium 102 is coupled totransmitters 104, 106, 108, and 110 and sensors 112, 114, 116, and 118.In various embodiments, the propagating medium includes one or more ofthe following: panel, table, glass, screen, door, floor, whiteboard,glass, plastic, wood, steel, metal, semiconductor, insulator, conductor,and any medium that is able to propagate an acoustic or ultrasonicsignal. For example, medium 102 is glass of a display screen. A firstsurface of medium 102 includes a surface area where a user may touch toprovide a selection input and a substantially opposite surface of medium102 is coupled to the transmitters and sensors shown in FIG. 1. Invarious embodiments, a surface of medium 102 is substantially flat,curved, or combinations thereof and may be configured in a variety ofshapes such as rectangular, square, oval, circular, trapezoidal,annular, or any combination of these, and the like.

Examples of transmitters 104, 106, 108, and 110 include piezoelectrictransducers, electromagnetic transducers, transmitters, sensors and/orany other transmitters and transducers capable of propagating a signalthrough medium 102. Examples of sensors 112, 114, 116, and 118 includepiezoelectric transducers, electromagnetic transducers, transmittersand/or any other sensors and transducers capable of detecting a signalon medium 102. In some embodiments, the transmitters and sensors shownin FIG. 1 are coupled to medium 102 in a manner that allows a user inputto be detected in a predetermined region of medium 102. Although fourtransmitters and four sensors are shown, any number of transmitters andany number of sensors may be used in other embodiments. For example, twotransmitters and three sensors may be used. In some embodiments, asingle transducer acts as both a transmitter and a sensor. For example,transmitter 104 and sensor 112 represent a single piezoelectrictransducer. In the example shown, transmitter 104 may propagate a signalthrough medium 102. Sensors 112, 114, 116, and 118 receive thepropagated signal. In another embodiment, the transmitters/sensors inFIG. 1 are attached to a flexible cable coupled to medium 102 via anencapsulant and/or glue material and/or fasteners.

Touch detector 120 is connected to the transmitters and sensors shown inFIG. 1. In some embodiments, detector 120 includes one or more of thefollowing: an integrated circuit chip, a printed circuit board, aprocessor, and other electrical components and connectors. Detector 120determines and sends a signal to be propagated by transmitters 104, 106,108, and 110. Detector 120 also receives the signal detected by sensors112, 114, 116, and 118. The received signals are processed by detector120 to determine whether a disturbance associated with a user input hasbeen detected at a location on a surface of medium 102 associated withthe disturbance. Detector 120 is in communication with applicationsystem 122. Application system 122 uses information provided by detector120. For example, application system 122 receives from detector 120 acoordinate associated with a user touch input that is used byapplication system 122 to control a software application of applicationsystem 122. In some embodiments, application system 122 includes aprocessor and/or memory/storage. In other embodiments, detector 120 andapplication system 122 are at least in part included/processed in asingle processor. An example of data provided by detector 120 toapplication system 122 includes one or more of the following associatedwith a user indication: a location coordinate of a surface of medium102, a gesture, simultaneous user indications (e.g., multi-touch input),a time, a status, a direction, a velocity, a force magnitude, aproximity magnitude, a pressure, a size, and other measurable or derivedinformation.

In some embodiments, a touch input is received at location 130 on asurface of medium 102. For example, a user touches the surface of medium102 at location 130. In some embodiments, one or more of transmitters104, 106, 108, and 110 transmit one or more active signals that arepropagated through medium 102. The touch input at location 130 disturbs(e.g., scatters) the propagated signal(s) and the disturbed signals arereceived at sensors 112, 114, 116, and 118. By measuring thedisturbance(s) of the propagated signal(s), the location and/or a forceassociated with the touch input may be determined.

FIG. 2 is a block diagram illustrating an embodiment of a system fordetecting a touch input. In some embodiments, touch detector 202 isincluded in touch detector 120 of FIG. 1. In some embodiments, thesystem of FIG. 2 is integrated in an integrated circuit chip. Touchdetector 202 includes system clock 204 that provides a synchronoussystem time source to one or more other components of detector 202.Controller 210 controls data flow and/or commands between microprocessor206, interface 208, DSP engine 220, and signal generator 212. In someembodiments, microprocessor 206 processes instructions and/orcalculations that can be used to program software/firmware and/orprocess data of detector 202. In some embodiments, a memory is coupledto microprocessor 206 and is configured to provide microprocessor 206with instructions. Signal generator 212 generates a signal to be used topropagate a signal such as a signal propagated by transmitter 104 ofFIG. 1. For example, signal generator 212 generates a pseudorandombinary sequence signal. Driver 214 receives the signal from generator212 and drives one or more transmitters, such as transmitters 104, 106,108, and 110 of FIG. 1, to propagate a signal through a medium.

A signal detected from a sensor such as sensor 112 of FIG. 1 is receivedby detector 202 and signal conditioner 216 conditions (e.g., filters)the received analog signal for further processing. For example, signalconditioner 216 receives the signal outputted by driver 214 and performsecho cancellation of the signal received by signal conditioner 216. Theconditioned signal is converted to a digital signal by analog-to-digitalconverter 218. The converted signal is processed by digital signalprocessor engine 220. For example, DSP engine 220 correlates theconverted signal against a reference signal. The result of thecorrelation may be used by microprocessor 206 to determine a locationassociated with a user touch input. In some embodiments, the DSP enginedetermines an amplitude change associated with the converted signal anda reference signal. The amplitude change may be used by microprocessor206 to determine a force associated with a user touch input. Interface208 provides an interface for microprocessor 206 and controller 210 thatallows an external component to access and/or control detector 202. Forexample, interface 208 allows detector 202 to communicate withapplication system 122 of FIG. 1 and provides the application systemwith location information associated with a user touch input.

FIG. 3 is a flow chart illustrating an embodiment of a process forcalibrating and validating touch detection. In some embodiments, theprocess of FIG. 3 is used at least in part to calibrate and validate thesystem of FIG. 1 and/or the system of FIG. 2. At 302, locations ofsignal transmitters and sensors with respect to a surface aredetermined. For example, locations of transmitters and sensors shown inFIG. 1 are determined with respect to their location on a surface ofmedium 102. In some embodiments, determining the locations includesreceiving location information. In various embodiments, one or more ofthe locations may be fixed and/or variable.

At 304, signal transmitters and sensors are calibrated. In someembodiments, calibrating the transmitter includes calibrating acharacteristic of a signal driver and/or transmitter (e.g., strength).In some embodiments, calibrating the sensor includes calibrating acharacteristic of a sensor (e.g., sensitivity). In some embodiments, thecalibration of 304 is performed to optimize the coverage and improvesignal-to-noise transmission/detection of a signal (e.g., acoustic orultrasonic) to be propagated through a medium and/or a disturbance to bedetected. For example, one or more components of the system of FIG. 1and/or the system of FIG. 2 are tuned to meet a signal-to-noiserequirement. In some embodiments, the calibration of 304 depends on thesize and type of a transmission/propagation medium and geometricconfiguration of the transmitters/sensors. In some embodiments, thecalibration of step 304 includes detecting a failure or aging of atransmitter or sensor. In some embodiments, the calibration of step 304includes cycling the transmitter and/or receiver. For example, toincrease the stability and reliability of a piezoelectric transmitterand/or receiver, a burn-in cycle is performed using a burn-in signal. Insome embodiments, the step of 304 includes configuring at least onesensing device within a vicinity of a predetermined spatial region tocapture an indication associated with a disturbance using the sensingdevice. The disturbance is caused in a selected portion of the inputsignal corresponding to a selection portion of the predetermined spatialregion.

At 306, surface disturbance detection is calibrated. In someembodiments, a test signal is propagated through a medium such as medium102 of FIG. 1 to determine an expected sensed signal when no disturbancehas been applied. In some embodiments, a test signal is propagatedthrough a medium to determine a sensed signal when one or morepredetermined disturbances (e.g., predetermined touch) are applied at apredetermined location. Using the sensed signal, one or more componentsmay be adjusted to calibrate the disturbance detection.

At 308, a validation of a touch detection system is performed. Forexample, the system of FIG. 1 and/or FIG. 2 is testing usingpredetermined disturbance patterns to determine detection accuracy,detection resolution, multi-touch detection, and/or response time. Ifthe validation fails, the process of FIG. 3 may be at least in partrepeated and/or one or more components may be adjusted before performinganother validation.

FIG. 4 is a flow chart illustrating an embodiment of a process fordetecting a user touch input. In some embodiments, the process of FIG. 4is at least in part implemented on touch detector 120 of FIG. 1 and/ortouch detector 202 of FIG. 2. At 402, a signal that can be used topropagate an active signal through a surface region is sent. In someembodiments, sending the signal includes driving (e.g., using driver 214of FIG. 2) a transmitter such as a transducer (e.g., transmitter 104 ofFIG. 1) to propagate an active signal (e.g., acoustic or ultrasonic)through a propagating medium with the surface region. In someembodiments, the signal includes a sequence selected to optimizeautocorrelation (e.g., resulting in narrow/short peak) of the signal.For example, the signal includes a Zadoff-Chu sequence. In someembodiments, the signal includes a pseudorandom binary sequence with orwithout modulation. In some embodiments, the propagated signal is anacoustic signal. In some embodiments, the propagated signal is anultrasonic signal (e.g., outside the range of human hearing). Forexample, the propagated signal is a signal above 20 kHz (e.g., withinthe range between 80 kHz to 100 kHz). In other embodiments, thepropagated signal may be within the range of human hearing. In someembodiments, by using the active signal, a user input on or near thesurface region can be detected by detecting disturbances in the activesignal when it is received by a sensor on the propagating medium. Byusing an active signal rather than merely listening passively for a usertouch indication on the surface, other vibrations and disturbances thatare not likely associated with a user touch indication can be moreeasily discerned/filtered out. In some embodiments, the active signal isused in addition to receiving a passive signal from a user input todetermine the user input.

At 404, the active signal that has been disturbed by a disturbance ofthe surface region is received. The disturbance may be associated with auser touch indication. In some embodiments, the disturbance causes theactive signal that is propagating through a medium to be attenuatedand/or delayed. In some embodiments, the disturbance in a selectedportion of the active signal corresponds to a location on the surfacethat has been indicated (e.g., touched) by a user.

At 406, the received signal is processed to at least in part determine alocation associated with the disturbance. In some embodiments,determining the location includes extracting a desired signal from thereceived signal at least in part by removing or reducing undesiredcomponents of the received signal such as disturbances caused byextraneous noise and vibrations not useful in detecting a touch input.In some embodiments, determining the location includes comparing thereceived signal to a reference signal that has not been affected by thedisturbance. The result of the comparison may be used with a result ofother comparisons performed using the reference signal and othersignal(s) received at a plurality of sensors. The location, in someembodiments, is a location (e.g., a location coordinate) on the surfaceregion where a user has provided a touch input. In addition todetermining the location, one or more of the following informationassociated with the disturbance may be determined at 406: a gesture,simultaneous user indications (e.g., multi-touch input), a time, astatus, a direction, a velocity, a force magnitude, a proximitymagnitude, a pressure, a size, and other measurable or derivedinformation. In some embodiments, the location is not determined at 406if a location cannot be determined using the received signal and/or thedisturbance is determined to be not associated with a user input.Information determined at 406 may be provided and/or outputted.

Although FIG. 4 shows receiving and processing an active signal that hasbeen disturbed, in some embodiments, a received signal has not beendisturbed by a touch input and the received signal is processed todetermine that a touch input has not been detected. An indication that atouch input has not been detected may be provided/outputted.

FIG. 5 is a flow chart illustrating an embodiment of a process fordetermining a location associated with a disturbance on a surface. Insome embodiments, the process of FIG. 5 is included in 406 of FIG. 4.The process of FIG. 5 may be implemented in touch detector 120 of FIG. 1and/or touch detector 202 of FIG. 2. At 502, a received signal isconditioned. In some embodiments, the received signal is a signalincluding a pseudorandom binary sequence that has been freely propagatedthrough a medium with a surface that can be used to receive a userinput. For example, the received signal is the signal that has beenreceived at 404 of FIG. 4. In some embodiments, conditioning the signalincludes filtering or otherwise modifying the received signal to improvesignal quality (e.g., signal-to-noise ratio) for detection of apseudorandom binary sequence included in the received signal and/or usertouch input. In some embodiments, conditioning the received signalincludes filtering out from the signal extraneous noise and/orvibrations not likely associated with a user touch indication.

At 504, an analog to digital signal conversion is performed on thesignal that has been conditioned at 502. In various embodiments, anynumber of standard analog to digital signal converters may be used. Theresulting digital signal is used to perform a first correlation at 506.In some embodiments, performing the first correlation includescorrelating the converted signal with a reference signal. Performing thecorrelation includes cross-correlating or determining a convolution(e.g., interferometry) of the converted signal with a reference signalto measure the similarity of the two signals as a time-lag is applied toone of the signals. By performing the correlation, the location of aportion of the converted signal that most corresponds to the referencesignal can be located. For example, a result of the correlation can beplotted as a graph of time within the received and converted signal(e.g., time-lag between the signals) vs. a measure of similarity. Theassociated time value of the largest value of the measure of similaritycorresponds to the location where the two signals most correspond. Bycomparing this measured time value against a reference time value (e.g.,at 306 of FIG. 3) not associated with a touch indication disturbance, atime delay/offset or phase difference caused on the received signal dueto a disturbance caused by a touch input can be determined. In someembodiments, by measuring the amplitude/intensity difference of thereceived signal at the determined time vs. a reference signal, a forceassociated with a touch indication may be determined. In someembodiments, the reference signal is determined based at least in parton the signal that was propagated through a medium (e.g., based on asource pseudorandom binary sequence signal that was propagated). In someembodiments, the reference signal is at least in part determined usinginformation determined during calibration at 306 of FIG. 3. Thereference signal may be chosen so that calculations required to beperformed during the correlation may be simplified. For example, thereference signal used in 506 is a simplified reference signal that canbe used to efficiently correlate the reference signal over a relativelylarge time difference (e.g., lag-time) between the received andconverted signal and the reference signal.

At 508, a second correlation is performed based on a result of the firstcorrelation. Performing the second correlation includes correlating(e.g., cross-correlation or convolution similar to step 506) theconverted signal in 504 with a second reference signal. The secondreference signal is a more complex/detailed (e.g., more computationallyintensive) reference signal as compared to the first reference signalused in 506. In some embodiments, the second correlation is performed in508 because using the second reference signal in 506 may be toocomputationally intensive for the time interval required to becorrelated in 506. Performing the second correlation based on the resultof the first correlation includes using one or more time valuesdetermined as a result of the first correlation. For example, using aresult of the first correlation, a range of likely time values (e.g.,time-lag) that most correlate between the received signal and the firstreference signal is determined and the second correlation is performedusing the second reference signal only across the determined range oftime values to fine tune and determine the time value that mostcorresponds to where the second reference signal (and, by association,also the first reference signal) matched the received signal. In variousembodiments, the first and second correlations have been used todetermine a portion within the received signal that correspond to adisturbance caused by a touch input at a location on a surface of apropagating medium. In other embodiments, the second correlation isoptional. For example, only a single correlation step is performed.

At 510, a result of the second correlation is used to at least in partdetermine a location associated with a disturbance. In some embodiments,determining the location includes comparing a determined time valuewhere the signals of the second correlation are most correlated andcomparing the determined time value with a reference time value (e.g.,determined at 306 of FIG. 3) not associated with a touch inputdisturbance, to determine a time delay/offset or phase difference causedon the received signal due to the disturbance (e.g., caused by a touchinput). This time delay is associated with a signal received at a firstsensor and other time delays due to the disturbance at other signalsreceived at other sensors are used to calculate a location of thedisturbance relative to the locations of the sensors. By using thelocation of the sensors relative to a surface of a medium that haspropagated the received signal, a location on the surface where thedisturbance originated may be determined.

FIG. 6 is a flowchart illustrating an embodiment of a process fordetermining a location associated with a disturbance. In someembodiments, the process of FIG. 6 is included in 510 of FIG. 5. At 602,a plurality of results of correlations performed on a plurality ofsignals disturbed by a disturbance of a surface is received. Forexample, a result of the correlation performed at 508 of FIG. 5 isreceived. In some embodiments, a signal is propagated using transmitter104 and sensors 114, 116, and 118 each receives the propagated signalthat has been disturbed by a touch input on or near a surface of medium102 of FIG. 1. The propagated signal may contain a predetermined signaland the predetermined signal is received at the various sensors. Each ofthe received signals is correlated with a reference signal to determinethe results received at 602. In some embodiments, the received resultsare associated with a same signal content (e.g., same binary sequence)that has been freely propagated on a medium at the same time. In someembodiments, the received results are associated with different signalcontents that have been disturbed by the same disturbance.

At 604, time differences associated with the plurality of results areused to determine a location associated with the disturbance. In someembodiments, each of the time differences is associated with a time whensignals used in the correlation are most correlated. In someembodiments, the time differences are associated with a determined timedelay/offset or phase difference caused on the received signal due tothe disturbance. This time delay may be calculated by comparing a timevalue determined using a correlation with a reference time value that isassociated with a scenario where a touch input has not been specified.The result of the comparison may be used to calculate a location of thedisturbance relative to the locations of sensors that received theplurality of signals. By using the location of the sensors relative to asurface of a medium that has propagated the received signal, a locationon the surface where the disturbance originated may be determined.

FIG. 7 is a flowchart illustrating an embodiment of a process ofdetermining a force associated with a user input. The process of FIG. 7may be implemented on touch detector 120 of FIG. 1 and/or touch detector202 of FIG. 2.

At 702, a location associated with a user input on a touch input surfaceis determined. In some embodiments, at least a portion of the process ofFIG. 4 is included in step 702. For example, the process of FIG. 4 isused to determine a location associated with a user touch input. Inanother example, a location associated with a user input at location 130on a surface of medium 102 of FIG. 1 is determined.

At 704, one or more received signals are selected to be evaluated. Insome embodiments, selecting the signal(s) to be evaluated includeselecting one or more desired signals from a plurality of receivedsignals used to detect the location associated with the user input. Forexample, one or more signals received in step 404 of FIG. 4 areselected. In some embodiments, the selected signal(s) are selected basedat least in part on a signal-to-noise ratio associated with signals. Insome embodiments, one or more signals with the highest signal-to-noiseratio are selected. For example, when an active signal that ispropagated through a touch input surface medium is disturbed/scatteredby a touch input, the disturbed signal is detected/received at variousdetectors/sensors/receivers coupled to the medium. The receiveddisturbed signals may be subject to other undesirable disturbances suchas other minor vibration sources (e.g., due to external audio vibration,device movement, etc.) that also disturb the active signal. The effectsof these undesirable disturbances may be larger on received signals thatwere received further away from the location of the touch input.

In some embodiments, a variation (e.g., disturbance such as amplitudechange) detected in an active signal received at a receiver/sensor maybe greater at certain receivers (e.g., receivers located closest to thelocation of the touch input) as compared to other receivers. Forexample, in the example of FIG. 1, touch input at location 130 disturbsan active signal sent by transmitter 104. The disturbed active signal isreceived at sensors/receivers 112, 114, 116, and 118. Becausesensor/receiver 114 is located closest to touch input location 130, ithas received a disturbed signal with the largest amplitude variationthat is proportional to the force of the touch input. In someembodiments, the selected signals may have been selected at least inpart by examining the amplitude of a detected disturbance. For example,one or more signals with the highest amplitude associated with adetected touch input disturbance are selected. In some embodiments,based at least in part on a location determined in 702, one or moresignals received at one or more receivers located closest to the touchinput location are selected. In some embodiments, a plurality of activesignals is used to detect a touch input location and/or touch inputforce intensity. One or more received signals to be used to determine aforce intensity may be selected for each of the active signals. In someembodiments, one or more received signals to be used to determine theforce intensity may be selected across the received signals of all theactive signals.

At 706, the one or more selected signals are normalized. In someembodiments, normalizing a selected signal includes adjusting (e.g.,scaling) an amplitude of the selected signal based on a distance valueassociated with the selected signal. For example, although anamount/intensity of force of a touch input may be detected by measuringan amplitude of a received active signal that has been disturbed by theforce of the touch input, other factors such as the location of thetouch input with respect to a receiver that has received the disturbedsignal and/or location of the transmitter transmitting the active signalmay also affect the amplitude of the received signal used to determinethe intensity of the force. In some embodiments, a distancevalue/identifier associated with one or more of the following is used todetermine a scaling factor used to scale a selected signal: a distancebetween a location of a touch input and a location of a receiver thathas received the selected signal, a distance between a location of atouch input and a location of a transmitter that has transmitted anactive signal that has been disturbed by a touch input and received asthe selected signal, a distance between a location of a receiver thathas received the selected signal and a location of a transmitter thathas transmitted an active signal that has been disturbed by a touchinput and received as the selected signal, and a combined distance of afirst distance between a location of a touch input and a location of areceiver that has received the selected signal and a second distancebetween the location of the touch input and a location of a transmitterthat has transmitted an active signal that has been disturbed by a touchinput and received as the selected signal. In some embodiments, each ofone or more selected signals is normalized by a different amount (e.g.,different amplitude scaling factors).

At 708, a force intensity identifier associated with the one or morenormalized signals is determined. The force intensity identifier mayinclude a numerical value and/or other identifier identifying a forceintensity. In some embodiments, if a plurality of normalized signals isused, an associated force may be determined for each normalized signaland the determined forces may be averaged and/or weighted-averaged todetermine the amount of the force. For example, in the case of weightedaveraging of the force values, each determined force value is weightedbased on an associated signal-to-noise ratio, an associated amplitudevalue, and/or an associated distance value between a receiver of thenormalized signal and the location of the touch input.

In some embodiments, the amount of force is determined using a measuredamplitude associated with a disturbed portion of the normalized signal.For example, the normalized signal represents a received active signalthat has been disturbed when a touch input was provided on a surface ofa medium that was propagating the active signal. A reference signal mayindicate a reference amplitude of a received active signal if the activesignal was not disturbed by a touch input. In some embodiments, anamplitude value associated with an amplitude change to the normalizedsignal caused by a force intensity of a touch input is determined. Forexample, the amplitude value may be a measured amplitude of adisturbance detected in a normalized signal or a difference between areference amplitude and the measured amplitude of the disturbancedetected in the normalized signal. In some embodiments, the amplitudevalue is used to obtain an amount/intensity of a force.

In some embodiments, the use of the amplitude value includes using theamplitude value to look up in a data structure (e.g., table, database,chart, graph, lookup table, list, etc.) a corresponding associated forceintensity. For example, the data structure includes entries associatinga signal disturbance amplitude value and a corresponding force intensityidentifier. The data structure may be predetermined/pre-computed. Forexample, for a given device, a controlled amount of force is applied andthe disturbance effect on an active signal due to the controlled amountof force is measured to determine an entry for the data structure. Theforce intensity may be varied to determine other entries of the datastructure. In some embodiments, the data structure is associated with aspecific receiver that received the signal included in the normalizedsignal. For example, the data structure includes data that has beenspecifically determined for characteristics of a specific receiver(e.g., for sensor/receiver 114 of FIG. 1). In some embodiments, the useof the amplitude value to look up a corresponding force intensityidentifier stored in a data structure includes selecting a specific datastructure and/or a specific portion of a data structure corresponding tothe normalized signal and/or a receiver that received the signalincluded in the normalized signal. In some embodiments, the datastructure is associated with a plurality of receivers. For example, thedata structure includes entries associated with averages of datadetermined for characteristics of each receiver in the plurality ofreceivers. In this example, the same data structure may be used for aplurality of normalized signals associated with various receivers.

In some embodiments, the use of the amplitude value includes using theamplitude value in a formula that can be used to simulate and/orcalculate a corresponding force intensity. For example, the amplitudevalue is used as an input to a predetermined formula used to compute acorresponding force intensity. In some embodiments, the formula isassociated with a specific receiver that received the signal of thenormalized signal. For example, the formula includes one or moreparameters (e.g., coefficients) that have been specifically determinedfor characteristics of a specific receiver (e.g., for sensor/receiver114 of FIG. 1). In some embodiments, the use of the amplitude value in aformula calculation includes selecting a specific formula correspondingto the normalized signal and/or a receiver that received the signalincluded in the normalized signal. In some embodiments, a single formulais associated with a plurality of receivers. For example, a formulaincludes averaged parameter values of parameter values that have beenspecifically determined for characteristics for each of the receivers inthe plurality of receivers. In this example, the same formula may beused for a plurality of normalized signals associated with differentreceivers.

At 710, the determined force intensity identifier is provided. In someembodiments, providing the force intensity identifier includes providingthe identifier (e.g., a numerical value, an identifier within a scale,etc.) to an application such as an application of application system 122of FIG. 1. In some embodiments, the provided force intensity identifieris provided with a corresponding touch input location identifierdetermined in step 406 of FIG. 4. In some embodiments, the providedforce intensity identifier is used to provide a user interfaceinteraction.

FIG. 8 is a flowchart illustrating an embodiment of a process fordetermining an entry of a data structure used to determine a forceintensity identifier. In some embodiments, the process of FIG. 8 isincluded in step 304 of FIG. 3. In some embodiments, the process of FIG.8 is used at least in part to create the data structure that may be usedin step 708 of FIG. 7. In some embodiments, the process of FIG. 8 isused at least in part to calibrate the system of FIG. 1 and/or thesystem of FIG. 2. In some embodiments, the process of FIG. 8 is used atleast in part to determine a data structure that can be included in oneor more devices to be manufactured to determine a force intensityidentifier/value corresponding to an amplitude value of a disturbancedetected in the received active signal. For example, the data structuremay be determined for a plurality of similar devices to be manufacturedor the data structure may be determined for a specific device takinginto account the manufacturing variation of the device.

At 802, a controlled amount of force is applied at a selected locationon a touch input surface. In some embodiments, the force is provided ona location of a surface of medium 102 of FIG. 1 where a touch input maybe provided. In some embodiments, a tip of a pointer (e.g., stylus) ispressing at the surface with a controllable amount of force. Forexample, a controlled amount of force is applied on a touch inputsurface while an active signal is being propagated through a medium ofthe touch input surface. The amount of force applied in 802 may be oneof a plurality of different amounts of force that will be applied on thetouch input surface.

At 804, an effect of the applied force is measured using one or morereceivers. Examples of the receivers include sensors 112-118 of FIG. 1and transducer transmitters used as receivers (e.g., transmitters104-110 of FIG. 1). In some embodiments, measuring the effect includesmeasuring an amplitude associated with a disturbed portion of an activesignal that has been disturbed when the force was applied in 802 andthat has been received by the one or more receivers. The amplitude maybe a directly measured amplitude value or a difference between areference amplitude and a detected amplitude. In some embodiments, thesignal received by the one or more receivers is normalized before theamplitude is measured. In some embodiments, normalizing a receivedsignal includes adjusting (e.g., scaling) an amplitude of the signalbased on a distance value associated with the selected signal.

A reference signal may indicate a reference amplitude of a receivedactive signal that has not been disturbed by a touch input. In someembodiments, an amplitude value associated with an amplitude changecaused by a disturbance of a touch input is determined. For example, theamplitude value may be a measured amplitude value of a disturbancedetected in a normalized signal or a difference between a referenceamplitude and the measured amplitude value of the disturbance detectedin the normalized signal. In some embodiments, the amplitude value isused to obtain an identifier of a force intensity.

In some embodiments, a distance value associated with one or more of thefollowing is used to determine a scaling factor used to scale a receivedsignal before an effect of a disturbance is measured using the receivedsignal: a distance between a location of a touch input and a location ofa receiver that has received the selected signal, a distance between alocation of the force input and a location of a transmitter that hastransmitted an active signal that has been disturbed by the force inputand received by the receiver, a distance between a location of thereceiver and a location of a transmitter that has transmitted an activesignal that has been disturbed by the force input and received by thereceiver, and a combined distance of a first distance between a locationof a force input and a location of the receiver and a second distancebetween the location of the force input and a location of a transmitterthat has transmitted an active signal that has been disturbed by theforce input and received by the receiver. In some embodiments, each ofone or more signals received by different receivers is normalized by adifferent amount (e.g., different amplitude scaling factors).

At 806, data associated with the measured effect is stored. In someembodiments, storing the data includes storing an entry in a datastructure such as the data structure that may be used in step 708 ofFIG. 7. For example, an entry that associates the amplitude valuedetermined in 804 and an identifier associated with an amount of forceapplied in 802 is stored in the data structure. In some embodiments,storing the data includes indexing the data by an amplitude valuedetermined in 804. For example, the stored data may be retrieved fromthe storage using the amplitude value. In some embodiments, the datastructure is determined for a specific signal receiver. In someembodiments, a data structure is determined for a plurality of signalreceivers. For example, data associated with the measured effect onsignals received at each receiver of a plurality of receivers isaveraged and stored. In some embodiments, storing the data includesstoring the data in a format that can be used to generate a graph suchas the graph of FIG. 9.

In some embodiments, the process of FIG. 8 is repeated for differentapplied force intensities, different receivers, different forceapplication locations, and/or different types of applied forces (e.g.,different force application tip). Data stored from the repeatedexecution of the steps of FIG. 8 may be used to fill the data structurethat may be used in step 708 of FIG. 7.

FIG. 9 includes graphs illustrating examples of a relationship between anormalized amplitude value of a measured disturbance and an appliedforce. Graph 900 plots an applied force intensity (in grams of force) ofa touch input vs. a measured amplitude of a disturbance caused by theapplied force for a single receiver. Graph 902 plots an applied forceintensity of a touch input vs. a measured amplitude of a disturbancecaused by the applied force for different receivers. The plots of thedifferent receivers may be averaged and combined into a single plot. Insome embodiments, graph 900 and/or graph 902 may be derived from datastored in the data structure that may be used in step 708 of FIG. 7. Insome embodiments, graph 900 and/or graph 902 may be generated using datastored in step 806 of FIG. 8. Graphs 900 and 902 show that there existsan increasing functional relationship between measured amplitude andapplied force. Using a predetermined graph, data structure, and/orformula that model this relationship, an associated force intensityidentifier may be determined for a given amplitude value (e.g., such asin step 708 of FIG. 7).

FIG. 10 is a flowchart illustrating an embodiment of a process forproviding a combined force. The process of FIG. 10 may be implemented ontouch detector 120 of FIG. 1 and/or touch detector 202 of FIG. 2.

At 1002, forces associated with each touch input location point of aplurality of touch input location points are determined. In someembodiments, a user touch input may be represented by a plurality oftouch input locations (e.g., multi-touch input, touch input covering arelatively large area, etc.). In some embodiments, for each touch inputlocation point, at least a portion of the process of FIG. 7 is used todetermine an associated force. For example, a force intensity identifieris determined for each input location in the plurality of touch inputlocations.

At 1004, the determined forces are combined to determine a combinedforce. For example, the combined force represents a total amount offorce applied on a touch input surface. In some embodiments, combiningthe forces includes adding a numerical representation of the forcestogether to determine the combined force. In some embodiments, anumerical representation of each determined force is weighted beforebeing added together. For example, each numerical value of a determinedforce is weighted (e.g., multiplied by a scalar) based on an associatedsignal-to-noise ratio, an associated amplitude value, and/or anassociated distance value between a receiver and a location of a touchinput. In some embodiments, the weights of the forces being weightedmust sum to the number of forces being combined.

At 1006, the combined force is provided. In some embodiments, providingthe combined force includes providing a force intensity identifier to anapplication such as an application of application system 122 of FIG. 1.In some embodiments, provided combined force is used to provide a userinterface interaction. In an alternative embodiment, rather thanproviding the combine force, the determined forces for each touch inputlocation point of a plurality of touch input location points areprovided.

FIG. 11 is a flowchart illustrating an embodiment of a process forproviding a user interface interaction. The process of FIG. 11 may beimplemented on touch detector 120 of FIG. 1 and/or touch detector 202 ofFIG. 2.

At 1102, one or more indicators associated with a location and a forceintensity of a user input are received. In some embodiments, theindicator(s) include data provided in step 710 of FIG. 7 and/or step1006 of FIG. 10. In some embodiments, indicators associated with asequence of locations and associated force intensities are received.

At 1104, a user interface object associated with the location isdetermined. In some embodiments, the user input is a touch screen userinterface input and the user interface element desired to be indicatedby the user input is determined. For example, the user input is detectedat a location where an icon has been displayed and it is determined thata user has desired to select the user icon by providing a touch input ata location of the icon. In some embodiments, the user interface objectincludes an object displayed on a touchscreen. In some embodiments, theuser interface object is not an object already displayed on a screen.For example, a hidden keyboard user interface object appears when a usertouches a specific area of a touch input screen.

At 1106, a user interface interaction based at least in part on the userinterface object and the force intensity is provided. For example, auser may indicate a desired user interface action by varying the amountof force applied on a touch input surface and the user interactionindicated by the received data in 1102 is provided. Examples of thepossible user interface interactions are described in the followingparagraphs.

FIG. 12 is a diagram showing an example user interface interaction usingforce information to drag and drop an item into a file system folder. Insome embodiments, a user may drag a desired item (e.g., a file, afolder, a reference, a link, an object, etc.) by touching the desireditem with a relatively “light” force applied to a pointer (e.g., finger,stylus, etc.) and dragging the pointer. A user may desire to drag anddrop the desired item to a folder to move or copy the item into thefolder. However if the user wants to drag and drop the desired item intoa subfolder of the folder, a user typically has to open the folder toreveal the desired subfolder before dragging and dropping the desireditem. In some embodiments, in order to move or copy an item to asubfolder of a displayed folder, a user may drag the desired item bytouching the desired item with a relatively “light” force applied to apointer (e.g., finger, stylus, etc.) and dragging the pointer to thedisplayed folder and applying a force intensity above a threshold levelto descend into the subfolders of the displayed folder and releasing thepointer once a desired subfolder is found. As shown in diagram 1200, afile may be moved by a “light” touch input to an icon representing thefile and dragging the touch input to a displayed folder and applyinggreater force intensity to the touch input to descend into the contentsof the displayed folder until a desired destination subfolder isdisplayed. In some embodiments, by varying the amount of pressure of atouch input, a file system hierarchy may be explored. In someembodiments, a touch input force intensity greater than a firstthreshold level indicates a command to navigate into a lower file systemhierarchy and a touch input force less than a second threshold level (insome cases, the second threshold level may be the same as the firstthreshold level) indicates a command to navigate to a higher file systemhierarchy. The threshold levels may be preconfigured, dynamicallydetermined, and/or may be configurable.

FIG. 13 is a diagram showing an example user interface interaction usingforce information to provide a context menu. In some embodiments,traditional touch input device button (e.g., mouse button) functionalitymay be mapped to one or more force intensity levels. For example, a“left button click” input may be performed by a touch input with a forcewithin a first intensity range and a “right button click” input may beperformed by a touch input with a force within a second intensity range.In some embodiments, a “middle button click” input may be performed by atouch input with a force within a third intensity range. In someembodiments, a user may select an area (e.g., spreadsheet cells) or textby performing a touch and drag operation with a force intensity below apredetermined threshold. Before the touch input is released, a user mayindicate that a context menu is desired (e.g., “right button click”) byincreasing the force of the touch input above apredetermined/dynamic/configurable threshold level. Diagram 1300 showstext selected using a touch and drag operation and a context menudisplayed when the force of the touch input was increased above apredetermined/dynamic/configurable threshold level.

FIG. 14 and FIG. 15 are diagrams showing examples of user interfaceinteractions using force information to navigate a menu. As shown indiagram 1400, a user may navigate a menu by touching and dragging atouch input to desired menu items. A user selects a menu item byincreasing the force of the touch input above a threshold level and auser cancels the menu by releasing the touch input without everincreasing the force of the touch input above the threshold level. Asshown in diagram 1500, a user can navigate a cascading menu by touchingand dragging a touch input to desired cascading menu items. A userselects a cascading menu item by increasing the force of the touch inputabove a threshold level and a user cancels the cascading menu byreleasing touch input without ever increasing the force of the touchinput above the threshold level. The threshold levels may bepreconfigured, dynamically determined, and/or configurable.

FIG. 16 is a diagram showing an example user interface interaction usingforce information to interact with a virtual keyboard. In someembodiments, the virtual keyboard includes a keyboard that is displayedon a screen or projected on a surface. In some embodiments, a touchinput key of a virtual keyboard is only registered as a key press if theforce of the touch input is above a threshold level or within a firstintensity range. For example, “lightly” resting fingers on a virtualkeyboard will not register key presses on the virtual keyboard and atouch input will only be registered a key press when a greater forceintensity is provided on the key of the virtual keyboard. This mayreduce spurious key press events (e.g., often generated simply due to afinger lightly brushing or contacting the surface). In some embodiments,alternate key functionality may be indicated based on a force of touchinput. For example, if a force of a touch input on a key is within afirst range, a lower case or normal character of the key is indicatedand if the force of the touch input is within a second range (e.g.,greater than the first range), then a shifted/capitalized character ofthe key is indicated. The threshold levels may be preconfigured,dynamically determined, and/or configurable.

In some embodiments, a touch input gesture and a force associated withthe gesture indicates that a virtual keyboard should be displayed and/ornot displayed. For example, when a predetermined number of distincttouch inputs are detected simultaneously (e.g., 4 or 5 fingers of eachhand resting on a touch input surface), a keyboard is displayed. In someembodiments, a displayed virtual keyboard is oriented and/or located ona screen based at least in part on one or more touch inputs received.For example, a virtual keyboard is oriented and placed on a touch inputdisplay surface such that when fingers of a user are rested on thesurface, the keys of the home row of the virtual keyboard are placedunder the location and orientation of the placed fingers of the user toplace the virtual keyboard in standard touch typing position withrespect to the user's fingers. The keyboard may be split to match theorientation of fingers of the user's two hands. Diagram 1600 shows avirtual keyboard that has been displayed for a user that has placedfingers of the user's left hand higher and angled out as compared tofingers of the user's right hand that has been placed lower in theopposite angle. In some embodiments, a touch input to a key of thevirtual keyboard of diagram 1600 is only registered as a keyboard keypress if the force of the touch input is above a threshold. Thethreshold levels may be preconfigured, dynamically determined, and/orconfigurable.

FIG. 17 and FIG. 18 are diagrams showing example user interfaceinteractions using force information to zoom and select user interfaceobjects. In some embodiments, force information is used to aid innavigating a dense array of objects on the screen (such as icons, keys,or several hyperlinks close by one another in a body of text). As shownin diagram 1700 and 1800, by “lightly” touching a touch input displaysurface, a region around the touch contact point is shown magnified on adisplay, indicating in greater detail what object on the screen isunderneath the contact point that is being used to provide touch input.When the object to be selected (e.g., icon in diagram 1700 and keyboardkey in diagram 1800) is underneath the contact point, increasing theforce of the touch input to a level greater than a predeterminedthreshold level (e.g., configurable) selects the object. In someembodiments, when a user selects an object and/or completes a userinterface action, a physical feedback (e.g., haptic feedback), visualfeedback, and/or audio feedback may be provided. In some embodiments,when a force of a touch input reaches a threshold level, a physicalfeedback (e.g., haptic feedback), visual feedback, and/or audio feedbackmay be provided.

In some embodiments, force information of touch input is used todistinguish between different gestures that otherwise might be identicalor very similar. For example, a swipe touchscreen gesture of a firstforce intensity within a first threshold range may be interpreted as ascrolling/panning indication and a swipe touchscreen gesture of a secondforce intensity within a second threshold range may be interpreted as a“change to the next window/tab” indication.

FIG. 19 is a graph illustrating an example of a relationship betweendetected touch input force and direction of change in audio volume.Graph 1900 shows that when a touch input is within a first intensityrange, volume is not changed, when a touch input is within a secondintensity range, volume decrease functionality is indicated, and when atouch input is within a third intensity range, volume increasefunctionality is indicated. In some embodiments, force information of atouch input is used to control audio volume level of a device. In someembodiments, volume is increased if a force of a touch input is above athreshold value (e.g., predetermined, dynamically determined, and/orconfigurable) and the volume is decreased if the force is below thethreshold value. In some embodiments, the touch input must be receivedin a specified area (e.g., displayed volume adjustment bar or adesignated area of a device for adjusting volume) to control the volume.In some embodiments, the rate of change of the volume is proportional toan amount of force applied in a touch input. In some embodiments, anaudio output destination is selected based at least in part on a forceof a touch input. For example, the audio is outputted to an earpiece ofa device with no touch input, and as a touch input is provided with anincreasing force that meets a threshold level, a speakerphone functionengages at a volume proportional to a detected force.

FIG. 20 is a diagram showing an example user interface interaction usingforce information to interact with a slider bar. In some embodiments, aslider bar may be used to indicate an intensity level or a time location(e.g., video position during playback). In some embodiments, whennavigating through a video sequence, a user wants the slider to movequickly to a particular time index/portion of the sequence, but thenmove with greater precision to focus on a particular scene or even asingle frame of video. Diagram 2000 shows a slider bar that can be movedby touching and dragging on the slider bar with a touch input.

In some embodiments, a speed or precision of slider bar movement usingtouch input dragging may be proportional to the force intensity level ofthe touch input. For example, a slider control moves with detailed/fineprecision when “light” pressure is applied but moves with coarse/fasterprecision when “harder” pressure is applied. In some embodiments, theslider bar may be moved with greater (e.g., fine or less granular)precision when a touch input force intensity within a first intensityrange is applied and moved with less (e.g., coarse or more granular)precision when a touch input force intensity within a second intensityrange is applied. The threshold levels may be preconfigured, dynamicallydetermined, and/or configurable.

In some embodiments, a velocity at which an object such as a finger orstylus contacts a touch input surface is used to control a userinterface. For example, video games, virtual musical instruments (drumsand pianos are two common examples), and other applications may utilizevelocity information to provide desired functionality. In someembodiments, measurement of contact velocity may be achieved bymeasuring the rate of change of the force. For example, if the touchforce changes at a given point from 0 to 0.5 pounds in 20 milliseconds,it can be inferred that the finger or other object impacted the touchinput screen at high velocity. On the other hand, a change in force from0 to 0.1 pounds in 100 milliseconds could be construed as a relativelylow velocity. Both the absolute measure of pressure and therate-of-change of pressure may be useful measures of information in userinterface design.

Although the foregoing embodiments have been described in some detailfor purposes of clarity of understanding, the invention is not limitedto the details provided. There are many alternative ways of implementingthe invention. The disclosed embodiments are illustrative and notrestrictive.

What is claimed is:
 1. A system for providing a user interfaceinteraction, comprising: a communication interface configured to:receive a first set of one or more indicators associated with a firstlocation on a touch input surface and a first force intensity of a touchinput provided on and in direct contact with the touch input surface,wherein the first force intensity is within a first threshold range andcorresponds to a first type of user interface indication, and receive asecond set of one or more indicators associated with a second locationon the touch input surface and a second force intensity of the touchinput that has been in continuous direct contact with the touch inputsurface at least since the touch input was provided at the firstlocation, wherein the second force intensity is within a secondthreshold range and corresponds to a user selection of a second userinterface object located at the second location; and a processorconfigured to: detect the first location at least in part by detecting adisturbance to an acoustic wave, wherein the acoustic wave is propagatedfreely in multiple directions by a same transmitter to a plurality ofsensors through a touch input region of a propagating medium of thetouch input surface and the disturbance is caused by the touch input,determine a value of the first force intensity based at least in part ona measurement of an amplitude associated with a disturbed portion of theacoustic wave disturbed by the touch input associated with the firstlocation of the touch input, wherein the same disturbed portion of thesame acoustic wave is used determine both the first location of thetouch input and the value of the first force intensity, determine afirst user interface object associated with the first location,determine that the touch input has been in continuous direct contactwith the touch input surface at least from when the touch input wasprovided at the first location to when the touch input was dragged tothe second location, determine that the second force intensitycorresponds to a different type of user interface indication than thefirst force intensity, wherein the second force intensity indicates aselection of the second user interface object with respect to the firstuser interface object, and provide the user interface interaction basedat least in part on the second user interface object with respect to thefirst user interface object, the determination that the touch input hasbeen in continuous direct contact, the determination that the secondforce intensity corresponds to a different type of user interfaceindication than the first force intensity, the first set of one or moreindicators, and the second set of one or more indicators.
 2. The systemof claim 1, wherein the touch input is associated with dragging thefirst user interface object to the second user interface object.
 3. Thesystem of claim 1, wherein the touch input is associated with draggingthe first user interface object from the first location to the secondlocation.
 4. The system of claim 1, wherein the first user interfaceobject is a file system object.
 5. The system of claim 1, wherein thefirst threshold range and the second threshold range are eachconfigurable.
 6. The system of claim 1, wherein the second userinterface object is a file directory object.
 7. The system of claim 1,wherein the user interface interaction is associated with descending afile system hierarchy of a file system folder represented by an iconlocated at the second location in the event the second force intensitymeets a threshold that was not met by the first force intensity.
 8. Thesystem of claim 1, wherein the second user interface object is a userinterface menu.
 9. The system of claim 1, wherein the touch input isassociated with navigating a user interface menu.
 10. The system ofclaim 1, wherein the user interface interaction is associated withselecting a menu item located at the second location in the event thesecond force intensity meets a threshold that was not met by the firstforce intensity.
 11. The system of claim 1, wherein the touch input isassociated with selecting content located within an area.
 12. The systemof claim 1, wherein the user interface interaction is associated withproviding a context menu associated with the second user interfaceobject in the event the second force intensity meets a threshold thatwas not met by the first force intensity.
 13. The system of claim 1,wherein the user interface interaction is associated with an objectmagnifying at least a portion of an area surrounding a current locationassociated with the touch input.
 14. The system of claim 1, wherein theuser interface interaction is associated with selecting the second userinterface object displayed at the second location in a magnified view inthe event the second force intensity meets a threshold that was not metby the first force intensity.
 15. The system of claim 1, wherein theuser interface interaction is associated with changing a window tab. 16.The system of claim 1, wherein the first user interface object includesa file folder.
 17. The system of claim 1, wherein the user interfaceinteraction is associated with descending a file directory.
 18. Thesystem of claim 1, wherein the first user interface object represents afile and the second user interface object represents a folder and avariation in applied force intensity corresponds to navigation of afolder hierarchy of the second user interface object.
 19. The system ofclaim 1, wherein the acoustic wave is propagated through the propagatingmedium without a reflective array.
 20. The system of claim 1, whereinthe first force intensity is determined based at least in part on adifference in amplitude between the acoustic wave and a reference signalunaffected by the disturbance.
 21. A method for providing a userinterface interaction, comprising: receiving a first set of one or moreindicators associated with a first location on a touch input surface anda first force intensity of a touch input provided on and in directcontact with the touch input surface, wherein the first force intensityis within a first threshold range and corresponds to a first type ofuser interface indication; determining a first user interface objectassociated with the first location; detecting the first location atleast in part by detecting a disturbance to an acoustic wave, whereinthe acoustic wave is propagated freely in multiple directions by a sametransmitter to a plurality of sensors through a touch input region of apropagating medium of the touch input surface and the disturbance iscaused by the touch input; determining a value of the first forceintensity based at least in part on a measurement of an amplitudeassociated with a disturbed portion of the acoustic wave disturbed bythe touch input associated with the first location of the touch input,wherein the same disturbed portion of the same acoustic wave is useddetermine both the first location of the touch input and the value ofthe first force intensity; receiving a second set of one or moreindicators associated with a second location on the touch input surfaceand a second force intensity of the touch input that has been incontinuous direct contact with the touch input surface at least sincethe touch input was provided at the first location, wherein the secondforce intensity is within a second threshold range and corresponds to auser selection of a second user interface object located at the secondlocation; determining that the touch input has been in continuous directcontact with the touch input surface at least from when the touch inputwas provided at the first location to when the touch input was draggedto the second location; determining that the second force intensitycorresponds to a different type of user interface indication than thefirst force intensity, wherein the second force intensity indicates aselection of the second user interface object with respect to the firstuser interface object; and using a processor to provide the userinterface interaction based at least in part on the second userinterface object with respect to the first user interface object, thedetermination that the touch input has been in continuous directcontact, the determination that the second force intensity correspondsto a different type of user interface indication than the first forceintensity, the first set of one or more indicators, and the second setof one or more indicators.
 22. A computer program product for providinga user interface interaction, the computer program product beingembodied in a non-transitory computer readable storage medium andcomprising computer instructions for: receiving a first set of one ormore indicators associated with a first location on a touch inputsurface and a first force intensity of a touch input provided on and indirect contact with the touch input surface, wherein the first forceintensity is within a first threshold range and corresponds to a firsttype of user interface indication; determining a first user interfaceobject associated with the first location; detecting the first locationat least in part by detecting a disturbance to an acoustic wave, whereinthe acoustic wave is propagated freely in multiple directions by a sametransmitter to a plurality of sensors through a touch input region of apropagating medium of the touch input surface and the disturbance iscaused by the touch input; determining a value of the first forceintensity based at least in part on a measurement of an amplitudeassociated with a disturbed portion of the acoustic wave disturbed bythe touch input associated with the first location of the touch input,wherein the same disturbed portion of the same acoustic wave is useddetermine both the first location of the touch input and the value ofthe first force intensity; receiving a second set of one or moreindicators associated with a second location on the touch input surfaceand a second force intensity of the touch input that has been incontinuous direct contact with the touch input surface at least sincethe touch input was provided at the first location, wherein the secondforce intensity is within a second threshold range and corresponds to auser selection of a second user interface object located at the secondlocation; determining that the touch input has been in continuous directcontact with the touch input surface at least from when the touch inputwas provided at the first location to when the touch input was draggedto the second location; determining that the second force intensitycorresponds to a different type of user interface indication than thefirst force intensity, wherein the second force intensity indicates aselection of the second user interface object with respect to the firstuser interface object; and providing the user interface interactionbased at least in part on the second user interface object with respectto the first user interface object, the determination that the touchinput has been in continuous direct contact, the determination that thesecond force intensity corresponds to a different type of user interfaceindication than the first force intensity, the first set of one or moreindicators, and the second set of one or more indicators.